Decouple inference preparation and execution #68
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The preparation of the inference workspace and the execution itself are refactored into separate rules. Closes #65.
Motivation
Nice extra: debugging inference becomes easier. Starting from the top-level directory, we can now run a single case with
Summary of changes:
inference_forecasterandinference_interpolatorprepare_inference_forecasterto set up a forecaster's inference workspace, including the updated config and the resourcesprepare_inference_interpolationto set up an interpolator's inference workspace, including the updated config, the resources and the parent forecaster's directory (note: this is also created when the "forecaster" is actually the analysis, but we create a placeholder - this is because we cannot conditionally add or remove an output from a rule)execute_inferencewhich will be the same for any kind of model executed withanemoi-inference run config.yamlcommon.smkso it can be used in a rule'srundirective>=0.8.0,<0.9.0